Certain transmissions, particularly real-time audio transmissions such as Voice over Internet Protocol (VoIP), are sensitive to the quality of the network connection over which they are being transmitted. Packet loss has greatest effect on voice quality. Packet loss may occur due to buffer overflow within the network, due to packets being deliberately discarded as a result of some congestion control scheme (e.g., Random Early Detection), or due to transmission errors. Several of the mechanisms that can lead to packet loss are of a transient nature, and hence the resulting packet loss is bursty in nature. A Markovian loss model suitably represents the distribution of packet loss in the Internet.
Jitter (or packet-delay variation) also has an effect on voice quality. However, the use of a jitter buffer compensates for jitter by increasing delay. Incoming packets are buffered and then read out at a constant rate; if packets are excessively late in arriving, then they are discarded. Jitter buffers are often adaptive and adjust their depth dynamically based on either the current packet-discard rate or current estimated jitter level.
R. Cox, and R. Perkins, Results of a Subjective Listening Test for G.711 with Frame Erasure Concealment, Committee contribution TIA1.7/99-016, compared the impact of random and bursty packet loss on G.711 and G.729A codecs. They found that for low packet-loss rates, a bursty distribution gave a higher subjective quality than a non-bursty distribution, whereas for high packet-loss rates, the converse was true. Bursty packet loss is far more detrimental to voice quality than random or evenly-distributed packet loss because packet-loss concealment algorithms are not capable of masking bursty packet loss.
Network engineers and administrators need to know how the network is performing and need to make adjustments that affect the quality of transmissions over the network. Endpoint terminals likewise need to know how the network is performing so that they can make adjustments that affect the transmission quality, such as the choice of a codec that is used to encode the transmission. Metrics that help the administrators and the endpoint terminals to determine the network quality and performance include packet loss rates and packet re-ordering rates. Most VoIP systems report packet loss. For example, the definition of the Real-time Transport Protocol (RTP) and its companion RTP Control Protocol (RTCP), which is the protocol that is used by most VoIP systems, defines a packet-loss percentage in a reporting period and a cumulative-loss metric. (See, Internet Engineering Task Force (IETF) Network Working Group Request for Comments: 3550, “RTP: A Transport Protocol for Real-Time Applications” (July 2003)).
Although these metrics give an indication of packet loss, they do not provide enough information because they do not describe the characteristics of the loss, such as its distribution. For example, there is no indication as to whether the loss is bursty (i.e., several contiguous packets are lost), evenly spaced (e.g., every xth packet is dropped), or random. These characteristics are important for determining the quality of VoIP transport systems; for example, bursty packet losses are far more detrimental to voice quality than either random or evenly-distributed packet losses. This is because packet-loss-concealment algorithms are not good at masking bursty packet losses. Also, these RTP metrics provide no information for re-ordered packets and their distribution.
Compared to network characteristics such as loss or delay, the dynamics of packet reordering are far less-well understood. A significant part of the problem is the lack of standard experimental techniques for measuring the phenomena. Previous reordering studies have used measurement tools that are inherently biased, such as ping, or methodologies that scale poorly, such as analyzing multi-site packet traces. The lack of a standard measurement methodology also has hampered the creation of a standard reordering metric. Consequently, the results between different studies can superficially vary by an order of magnitude, thereby creating significant confusion and controversy about the prevalence of reordering.
One of the techniques for measuring packet reordering is to generate estimates by sending repeated ICMP echo-request packets to a remote host and then evaluating the order of the ICMP echo-reply packets that are generated in response. The benefit of this approach is that it allows paths to arbitrary hosts to be measured, so long as the end host will respond to ICMP requests. The most obvious limitations of this approach arise from the use of the ICMP echo request/reply protocol for measurement. Using this method, it is not possible to distinguish if a packet was reordered before it arrived at the remote host or after it left the remote host. Consequently, the measurements produced can both underestimate the total reordering and overestimate the re-ordering in either direction. Since most protocols are more sensitive to reordering in one direction than another, this ambiguity can be quite important. Also, the use of ICMP as a fine-grained measurement tool is problematic since system and network operators alike increasingly filter and rate-limit such traffic to address security concerns.
Although packet re-ordering metrics have been proposed for inclusion in Internet Protocol Performance Metrics (IPPM) (see, IETF Network Working Group “Packet Reordering Metric for IPPM” (March 2002)), very few VoIP systems presently log or report any information on packet re-ordering. This is principally because such reports require logging and maintaining of large amounts of historical data, which is a problem for low-cost devices, such as VoIP phones. It is also a burden for devices that seek to maximize channel density with the least amount of hardware, and for devices that have limited processing power and data memory to process such large amounts of historical data.